N93-1 1935 


FUZZY LOGIC CONTROL FOR CAMERA 
TRACKING SYSTEM 


Robert N. Lea 
Johnson Space Center 

J. Giarratano 

University of Houston Clear Lake 


ABSTRACT : 

A concept utilizing fuzzy theory has been 
developed for a camera tracking system to provide 
support for proximity operations and traffic 
management around the Space Station Freedom. 
Fuzzy sets and fuzzy logic based reasoning are used 
in a control system which utilizes images from a 
camera and generates required pan and tilt 
commands to track and maintain a moving target in 
the camera's field of view. This control system can 
be implemented on a fuzzy chip to provide an 
intelligent sensor for autonomous operations. 
Capabilities of the control system can be expanded 
to include approach, handover to other sensors, 
caution and warning messages. 

L INTRODUCTION 

Advanced sensor systems with intelligence 
and a distributed nature will be required for 
activities such as proximity operations and traffic 
control around the Space Station Freedom (SSF). 
These systems will receive various types of 
measurements from multiple sensors and perform 
the necessary data fusion for the Navigation, 
Guidance and Control systems. The SSF operational 
requirements necessitate that this system be 
composed of passive type low power sensors. (Based 
on the current design, the SSF operations are 
expected to be power limited and computing 
resources limited.) An important feature is that the 
system should be capable of handling imprecise and 
approximate measurements as well as sensor 
failures. 

A number of theories dealing with 
approximate reasoning such as fuzzy logic and 
Dempster-Shafer theory, were considered for this 
work and fuzzy logic was selected. Since its 
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inception by Lotfi Zadeh [1, 2] in the 1960 r s, fuzzy 
logic has been applied to many fields [3, 4, 5], to 
handle imprecise measurements. Applications of 
fuzzy logic have also been developed for the star- 
tracking system of the Space Shuttle [61, the attitude 
control [7] and a combined translational and 
rotational control of a spacecraft [8]. 

The concept development for such an 
advanced sensor system that can accept 
measurements from several cameras and laser 
rangefinders is in progress within the Software 
Technology Laboratory of the Information Systems 
Division at the Johnson Space Center. The first 
phase of this development is a Camera Tracking 
System based on a fuzzy logic approach that utilizes 
the object's pixel position and range as inputs and 
controls the camera gimbal drives to keep this 
object in the Field Of View (FOV) of the camera. 
Later phases will involve development of other 
functions as described in the future activity section. 

In this paper, we describe our concept of a 
fuzzy logic based tracking controller that meets 
these requirements. A typical proximity operations 
scenario and the tasks of a tracking system are 
discussed in section 2. The fuzzy logic approach to 
the tracking controller and details of the 
membership functions and rulebase are described in 
section 3. Advantages of the fuzzy logic approach 
for such a system are discussed in section 4. Future 
activities and a summary are given in section 5. 

2. OPERATIONS SCENARIO 

The proximity operations zone around the 
SSF is defined as a sphere of about 2000 feet. 
Within this zone, several activities [9] such as fly- 
around, final approach and docking will be 
performed involving other vehicles such as a Space 
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Shuttle, satellite servicer and other payloads. The 
zone also includes extravehicular activities 
performed by the crew in spacesuits or by 
telerobots to inspect the SSF structure and 
subsystems and, if necessary, install and/or replace 
the components. Some of the payloads approaching 
the SSF will require extensive involvement by the 
crew. All extra-vehicular activities, trajectories of 
incoming vehicles and all docking and berthing 
operations must be closely monitored to avoid 
collisions. 

A typical scenario for proximity and docking 
operations with the SSF is shown in fig. 1. Assume 
there is an autonomous satellite approaching the 
station from about 2000 feet. It first moves to about 
200 feet, then performs station keeping at that point 
for a short period of time. It then completes the 
necessary information exchange with the SSF, and 
proceeds to a closer point in such a way that the 
mobile remote manipulator system moving on the 
SSF structure can grapple it. This autonomous 
satellite, if necessary, can also perform a 'fly- 
around' to observe the SSF from various angles. 
During all these activities, cameras mounted on the 
SSF monitor the motion of the satellite and advise 
the crew in case of malfunction or irregularities. 
High power sensors like radar and lasers are less 
desirable when operations are power limited. 



FIG. 1 OPERATIONS SCENARIO FOR TRACKING TASKS 


Tracking of an object means aligning the 
pointing axis of a camera along the object's line-of- 
sight. vector The monitoring camera is typically 
mounted on the pan and tilt gimble drives which 
are capable of rotating the pointing axis within a 
certain range. The task of the tracking controller is 
to command these gimble drives so that the pointing 
axis of the camera is along the line-of-sight vector 
which is estimated from the measurements. 


There are several issues the tracking 
controller must handle properly while commanding 
the gimble drives. Since the objects could be fast or 
slow moving, the controller must keep track of the 
rotating speed properly. Furthermore, the image 
could be blurry with a low resolution so that the 
estimate of the line-of-sight vector is poor. 
Handover to another camera tracking system may 
be required because of physical limitations of 
gimble drives and its mounting geometry. The 
controller must be aware of its gimble limits and 
whether the drives are approaching these limits. 

There are two methods of commanding these 
gimble drives: 1) Rotate the gimble by the desired 
delta angle, and 2) Achieve the desired angular rate 
for that axis. In the tracking system under 
development, the gimble drives will be commanded 
using the second method. The object's line-of-sight 
vector will be estimated from the position 
measurements in terms of pixels ( as shown in fig. 
2) and is input to the control system. There is a 
laser rangefinder that provides a range 
measurement for the object in the FOV. This 
information can be used to properly estimate the 
effects of the objects speed on the rotation of the 
line-of-sight vector. 


CAMERA FIELD-OF-VTEW 
170 pixels 



Fig. 2 CONCEPT OF A CAMERA TRACKING SYSTEM 


3. FUZZY LOGIC APPROACH TO CONTROL 

For the fuzzy logic based tracking controller, 
the inputs are range and line-of-sight vector, and 
the outputs are the commanded pan and tilt rates. 
The line-of-sight vector is input in terms of pixel 
position in the camera FOV. When an image is 
received, it is processed to determine the location 
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of the object in the camera frame which has the 
vertical, horizontal and pointing vectors as three 
axes. Usually an image, particularly for complex 
objects, spans over many pixels. Using a suitable 
technique, the centroid of the image is computed as 
the current location in the viewing plane which is 
like a Cartesian coordinate plane having vertical 
and horizontal axes. The size of the viewing plane 
is 170 x 170 pixels, and the origin is at the upper 
left comer as shown in fig. 2. The range of the 
object is received from the laser rangefinder as a 
measurement. These three parameter values are 
input to the controller. 

Each of these parameters have their 
respective membership functions as shown in fig. 3. 
The range membership functions are Very_Far 
(VFAR), FAR, NEAR, Very_Near (VNEAR) and 
Proximity (PROX). The Proximity membership 
function represents a close proximity threshold 
within which docking operations can begin. The 
universe of discourse for range is from 0 to 200 
feet. If the spacecraft operations begin at 2000 feet, 
then this universe of discourse can be extended to 
that value, however, a large part is then contained 
in only one membership function, VFAR. 


RANGE MEMBERSHIP FUNCTIONS 


VFAR FAR NEAR VNEAR PROX 



MEMBERSHIP FUNCTIONS FOR HORIZONTAL POSITION 
FL LL CENTER LR FR 



0 20 30 42 53 73 8 5 97 116 128 140 150 170 


MEMBERSHIP FUNCTIONS FOR VERTICAL POSITION 


FU LU CENTER LD FD 



0 20 30 42 53 73 B5 97 116 128 140 150 170 


Fig. 3 Membership functions For the input parameters 


The horizontal position membership 
functions are Far_Left (FL), Little_Left (LL), 
CENTER, Little_Right (LR), and Far_Right (FR). 
The vertical position membership functions are 
Far_Up (FU), Little_Up (LU), CENTER, 
Little_Down (LD), and Far_Down (FD). The 
universe of discourse for the horizontal and vertical 


pixel positions is from 0 to 170. The typical FOV is 
about 45 degrees wide, however, the range of a 
gimble drive can be from -180 to 180 degrees and 
thus it should be noted that the FOV range does not 
represent the gimble drive range. 

The membership functions shown in fig. 4 
for the pan and tilt rates are the same with a 
universe of discourse from -6.0 to 6.0 degrees per 
second implying that the maximum rate that can be 
achieved by gimble drives is 6.0 degrees per second 
in one direction. Membership functions are Fast 
Negative (FN), Slow Negative (SN), Zero (ZR), 
Slow Positive (SP) and Fast Positive (FP). To 
properly handle the different ranges and effects of 
the velocity of objects in the FOV, there is a need 
to generate a scale factor. If the objects range is 
very far, then, with a typical approach velocity, its 
line-of-sight vector is not going to rotate fast. Thus, 
the pointing vector rotation can be managed with a 
low rate. If the object is very near or in the 
proximity range, then the approach velocity may 
rotate the line-of-sight vector with a high rate. In 
that case, the object may go out of the FOV 
quickly. In order to keep up with high angular rate 
of line-of-sight vector, the controller will need a 
high rate. The scale factor membership functions, 
shown in fig. 4, provide the capability to change the 
response based on the range measurements. All 
membership functions are triangular shaped for 
simplicity in implementation. 


MEMBERSHIP FUNCTIONS FOR S C ALE_F A CTOR 


HIGH 

MEDIUM 

LOW 

... N 
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PAN AND TILT RATE MEMBERSHIP FUNCTIONS 
FN SN ZR SP FP 



-6.0 -5D -3.0 0.0 3.0 5.0 6.0 


Fig. 4 Membership functions for the output parameters 


The desired image location is the center of 
the viewing plane, which is at (85,85). If the 
current location is close to the center, then rotation 
of the pointing axis is not required. If the location 
is to the left of center then a left rotation is 
necessary. Similarly, if the image is down from the 
horizontal line then a downward rotation is 
required. These rotations are determined using the 
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position and range measurements and the rulebase 
shown in Table I. First the range measurement is 
fuzzified and the value of the scale factor is 
determined based on the scale_factor rules. The 
necessary defuzzification processing is performed 
to compute the crisp value of the scale factor. Then, 
the crisp scale factor and the position measurements 
are provided to the next set of rules to determine 
the rate at which the gimble drives should be 
rotated. There are 30 rules that determine both pan 
and tilt rates. Again, the necessary defuzzification 
processing is performed to compute the crisp values 
of the pan and tilt rates which can be sent to the 
gimble drives as command values. 


Table I. Rule base for the tracking task 


] Distance Membership Functions j 
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VNEAR PROX | 
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MED 
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Horizontal Position Membership Functions 
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Pan_Rate Membership Functions 


Vertical Position Membership Functions 

FD LD CENTER LU RJ 


LOW 

FP 

SP 

ZR 

SN 

FN 

MED 

SP 

SP 

ZR 

SN 

SN 
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SP 

ZR 

ZR 

ZR 

SN 


Tilt_Rale Membership Functions 


Note - Negative Tilt_rate means the pointing axis going upward in FO V 


KEY : VFAR - Very Far, VNEAR - Very Near, PROX - Proxomity zone 
FL - Far Left, LL - Little Left, LR - Little Right, FR - Far Right 
FU - Far Up, LU - Little Up, LD - Little Down, FD - Far Down 
FN - Fast Negative, SN - Slow Negative, ZR - Zero, 

FP - Fast Positive, SP - Slow Positive 


The camera is moved based on these 
commands within the limits of its gimble rates and 
angles. New measurements in the camera FOV are 
obtained for the next cycle and the processing is 
repeated. The cycle time is based on the time 


requirements for the following functions: 1) 
determine pixel positions, 2) obtain a range 
measurement, 3) rotate the gimble drives at a 
desired rate, and 4) the requirements to track the 
object within a certain performance envelope. 
Typical cycle time ranges between 0.1 to 1.0 
second. 

4. ADVANTAGES OF FUZZY LOGIC 
APPROACH 

There are several advantages of our approach 
that utilizes fuzzy logic in a camera tracking 
system. The fuzzy logic approach is simple to 
understand and easy to implement as a software 
module. Fuzzy rules provide a framework to 
implement the human thinking process i.e. the rules 
reflect the human thought process, such as " If the 
object is Far_Left then rotate the camera to the left 
side The entire rule base is derived as if a human 
was performing the tracking task. 

Implementation of fuzzy membership 
functions, rules and related processing is made easy 
by tools like the TIL Shell [10] which has a 
graphics oriented user interface and fuzzy-C 
compilers [11] that can generate code for the fuzzy 
chip or the C code to integrate with other software 
modules. There are several commercial products 
available in the industry. 

It is also possible to develop and implement 
the fuzzy controller in the 'fuzzy processors', thus, 
having a fuzzy hardware controller. There are 
several commercial fuzzy processors [12, 13, 14] 
that can process over 30,000 fuzzy rules per second 
and thus provide a high speed processing power. 
These fuzzy processors consume low power with a 
capability to process general purpose instructions 
and can be mounted in the back plane of a camera. 
These processors also provide interfaces to 
hardware to transfer information and commands. 
Advanced sensor system envisioned for space 
station operations will have such processors 
embedded as an integral part of the system. 

The camera tracking system described here 
can be built as an intelligent sensor with built-in 
intelligence and speed to perform functions which 
are normally performed in the distributed 
processing network of approximately 20 computers 
(the actual number of computers may change with 
requirements) onboard SSF. Because of a dedicated 
fuzzy chip and its processing power, there is 
virtually no computational load to the SSF 
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computing network. As a result, the SSF computers 
will be available for other computing requirements 
such as complex guidance and navigation schemes. 
Furthermore, the interfaces between the fuzzy chip 
and the computing network will be at a command 
level requiring reasonably low speed data transfer. 
There will be no need for a high rate data transfer 
which can increase the cost and decrease the 
reliability. 

This system will involve low power sensors 
as compared to an active sensor e.g. Radar in Ku 
band range, or LADAR using laser frequency. 
Typically, the active sensor radiates a power pulse 
towards a target and receives back a reflected pulse. 
Based on the power transmitted, power received 
and time between these pulses, parameters like 
range and range rates are calculated. Since the 
camera tracking system will not be radiating 
power, it will be a low power system in comparison 
with active sensor system. Since there is already a 
shortage of power, an important consumable, 
onboard the SSF, availability of low power sensors 
is very important for continuous operations. The 
SSF can afford to keep this type of a sensor 
working around the clock without having much 
impact on the power management or other 
computational load on the main computers. 


In later phases of the development, it is 
planned to expand the concept and fuzzy logic 
approach from tracking to : 1) identifying potential 
sensor failures and notifying the crew 2) handover 
techniques for switching to a different type of 
sensor measurement, 3) identifying one or more 
objects appearing in the camera's FOV (object 
extraction and identification algorithms), and 4) 
traffic management guidance. For configurations 
where the camera is mounted on a movable 
platform or on an end-effector of a robotic arm, 
this concept will be expanded to perform tasks such 
as the approach towards the docking fixture of the 
object, and grappling and rigidizing operations 
(similar to the operations performed by the crew). 
The concept can also be expanded to incorporate 
sensor data fusion required for the debris avoidance 
task. 
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